Update Database

This section explains how to update the SysWESM database when the Historical Load Step 6 identifies errors in the Historical Database Error Report (HistDBErr.csv).

• Required Inputs
• Step-by-step procedure
• Exercise

Updating the SysWESM Database

The Historical Database Error file (HistDBErr.csv) is generated in the scenario folder when you run a historical load. It provides a summary of system errors including capacity upgrades, missing units in the database, out-of-merit no bid, out-of-merit outage, and link outage. This file can be viewed in Apollo by right-clicking a scenario or by navigating directly to the historical scenario folder.

An up-to-date system database is crucial for accurate Short-Term Forecasting, as the WESM constantly changes (for example, new units being commissioned or existing unit capacity upgrades).

Prerequisites

• Completed Historical Load run
• HistDBErr.csv generated in the scenario folder
• For the CSV method: Units.csv, Sites.csv, and/or Stors.csv template files prepared
• For the Database Edit Utility method: access to the SysWESMXXX.db database

Historical Database Errors

The Historical Database Error Report lists five categories of error:

• Capacity Upgrade — The capacity of an existing unit has changed. Suite automatically corrects this during the historical load process. To verify, rerun the historical load after the update.

• missing in db — Units, sites, or storage entries exist in the IEMOP data but are not yet in the SysWESM database. Add these using either the CSV method or the Database Edit Utility (see below).

• Out-of-Merit No Bid — Units with missing bids. These are included in backcast simulations using the generated bid file OM_XXX.bid from the historical load process.

• Out-of-Merit Outage — Unit outage events. These are included in backcast simulations using the generated event file Ev_XXX.csv from the historical load process.

• Line Outage — Line outage events. These are also captured in Ev_XXX.csv from the historical load process.

note

Before proceeding to Backcasting, ensure that the historical load has been completed and that the only remaining Historical Database Errors are Out-of-Merit No Bid, Out-of-Merit Outage, and/or Line Outage. Capacity Upgrade and missing-in-db errors must be resolved first.

Step-by-Step Guide

• Phase 1: Using CSV Files
• Phase 2: Using Database Edit Utility

1

Prepare CSV Files

Prepare the .csv files using the provided Units.csv, Sites.csv, and Stors.csv files as base templates.

At minimum, include the necessary details such as IDs, capacity, fuel type, and operations. Make sure that all details are filled out correctly.

2

Open the Database Editor

Open Suite, then go to the Database Editor tab.

3

Open the System Database

Click Open Database and select the system database file: SysWESMXXX.db

4

Add Units, Sites, or Storages

Click any item from the tree view, then click Add Units/Sites/Storages.

5

Select Input Files and Update

Select the correct input file(s) and output path(s), then click Update Database.

6

Confirm the Update

A confirmation message will appear. Before proceeding, make sure that the details for the new units, sites, and/or storages in the .csv files are correct.

7

Review the Updated Fields

After the update is complete, review the updated fields in the Database Editor to confirm that all details were added correctly.

---

Primary CSV Files

See each tab to learn more about the primary CSV files used when adding entries via the CSV method.

• Storages
• Sites
• Units

Storages.csv

The Storages.csv file defines the physical limits of the energy reservoir. It sets the maximum and minimum energy capacity (MWh) and the PcBeg (Beginning %), which determines the initial state of charge at the start of the simulation.

For Battery Energy Storage Systems (BESS), PcBeg is typically set to 50% to provide balanced charging and discharging flexibility.

The table below describes the parameters for storage assets (e.g., STO_CLUZ) as defined in the Storages.csv data structure.

Item	Description	Example Value
StorageID	Unique identifier of the storage.	STO_CLUZ (user-defined)
Region	Region associated with the storage.	CLUZ, CVIS, or CMIN
StorageType	Classification of the storage.	BAT, HYDRO, GAS
InFlow	Rate of inflow into the storage (for hydro or similar technologies).	0 (for BESS)
Max Cap (MWh)	Maximum energy capacity of the storage (MWh).	4000 (user-defined)
Min Cap (MWh)	Minimum allowable energy level (MWh).	1 (user-defined)
Begin Level (%)	Initial state of charge at the start of the simulation.	50 (user-defined)
MoF0 – MoF12	Monthly inflow values (if applicable).	0 (for BESS)
Upstream StorageID	Upstream storage ID (if applicable).	(No input required for BESS)
Upstream Factor	Upstream factor.	1
Inflow Time Hrs	Inflow time (hours).	0 (for BESS)

Exercise

Give it a try and do this exercise:

Step 1 of 3

Review the Historical Database Error Report

Open Apollo and right-click the historical scenario

Open HistDBErr.csv and review the error types

Note any 'missing in db' or 'Capacity Upgrade' errors that need attention

---

Suite Database Editor — Update Database

The Update Database modal applies bulk structural updates to the currently open database. Select an update mode, prepare your input files, and click Update to write changes directly to the database.

Opening Update Database

Click Update Database in the top-right area of the toolbar. The button is disabled when no database is open.

Update Modes

Clicking Update writes changes directly to the database — this cannot be undone via the main editor's Undo. When saving to the current file, any unsaved edits in the main editor are discarded. Save your work before clicking Update.

Capacity Upgrade

Capacity Upgrade reads a Historical Database Error Report and updates Units.MwMax for each unit whose Remarks column equals "capacity upgrade". Only one column in one table is changed — no rows are inserted.

The loaded database must have Sites, Units, and Storages tables (the USS schema). If the schema check fails, an error banner appears and the update is refused.

• Available on: Desktop only
• Web display: The file pickers still appear, but clicking Update returns a "not yet available" error banner.

Select a Historical Database Error Report file (.csv, .xlsx, or .xls) using the Historical Database Error Report (.csv) picker. Your file must include the following nine columns (case-sensitive; column order does not matter):

• UnitID — matched against Units.DUID to identify which row to update
• MaxGen — new value written to Units.MwMax; must be a positive number
• Remarks — only rows where this equals "capacity upgrade" are processed; all others are skipped
• Required for file identification but not used by the update logic: SiteID, RegionID, MaxCap, MREnergy, FuelType, CapDeficit

Extra columns are tolerated. If any required column is missing, the update is refused with a red error banner naming the missing columns. If characters appear garbled, confirm the file is saved in UTF-8 encoding.

The modal prompts for an output path:

• Same path as the open database — saves into the current database file; any unsaved edits in the main editor are discarded.
• A different path — saves a copy (Save As); any unsaved edits in the main editor are saved into the copy before the update runs.

After clicking Update:

• Green block: N unit(s) upgraded in Units with a list of DUID: old → new MW values.
• Neutral block: No capacity upgrade rows were found — valid file, but no rows matched "capacity upgrade" in Remarks; Units was not changed.
• Orange warning block: N CSV row(s) had no matching unit — DUIDs in the file that did not match any unit in the database.
• Warning list: row skipped — for non-numeric or non-positive MaxGen values.
• Red error banner: missing columns, wrong file type, or USS schema not found.

Generation Files

Generation Files reconciles a database against two external data sources — an MNM GenList file and one or more CAPEG files — performing up to ten categories of structural update in one pass.

Generation Files requires a regional database — the Regions table must contain exactly CLUZ, CVIS, and CMIN as RegIDs. If this precondition is not met, the modal replaces the mode option with a disabled note and the mode cannot be selected.

Generation Files is not idempotent

Running Generation Files more than once on the same database with the same input files will still produce changes on subsequent runs. Measured results on a real database:

• Run 1: 1,271 changes
• Run 2: 80 changes
• Run 3: 3 changes

Do not re-run Generation Files casually. Each run may modify the database.

• Available on: Desktop only
• Web display: Clicking Update shows "This operation is not yet implemented."

Select an MNM GenList file and one or more CAPEG files (.csv, .xlsx, or .xls each). Both files must end with an EOF row — a row where the first column reads EOF and all remaining columns are empty.

MNM GenList required columns (case-sensitive): RESOURCE NAME, DESCRIPTION, REGION NAME, STATION NAME, and TRADING PARTICIPANT. Region names in the MNM GenList file use the full names LUZON, VISAYAS, and MINDANAO; these are automatically mapped to the database's region codes CLUZ, CVIS, and CMIN. If characters appear garbled, confirm the file is saved in Windows-1252 (CP1252) encoding — MNM GenList files are read with that encoding.

CAPEG required columns (case-sensitive): TP_NAME, RESOURCE_NAME, and MAXIMUM_CAPACITY. TRADING_DAY_ID is optional — when multiple CAPEG files are provided, the row with the largest TRADING_DAY_ID wins for each resource name (ISO date format means lexicographic order equals chronological order). CAPEG files are read as latin1; files that contain only ASCII characters will not be affected.

In a single pass, Generation Files may perform any of the following:

• Refresh Units.Name for units whose description changed
• Raise Units.MwMax when the CAPEG MAXIMUM_CAPACITY exceeds the current value
• Insert new Units (cloned from a same-fuel-type donor row, inheriting MLF and MTYR)
• Insert new Sites (if STATION NAME does not already exist as a SiteID)
• Insert missing Portfolio (Parts) entries from TRADING PARTICIPANT
• Recompute Sites.Max and Sites.Cap for all sites with at least one child unit
• Rename Sites to add a fuel-type suffix (for example, 01LAMAO becomes 01LAMAO_C for coal)
• Create load-twin Sites for battery (_B/_BL) and pump (_P/_PL) fuel types
• Sync pre-existing load-twin Sites (gen-wins: overwrites load twin with values from the gen site; reported as a warning)
• Normalize malformed twin DUIDs

When a unit's fuel type cannot be identified, sites are currently left unsuffixed — this behaviour is pending final confirmation and may change in a future release.

The modal prompts for an output path:

• Same path as the open database — saves into the current database file; any unsaved edits in the main editor are discarded.
• A different path — saves a copy (Save As); any unsaved edits in the main editor are saved into the copy before the update runs.